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ABSTRACT 

For many years assessment strategies and practices have emphasized on the one hand the importance of integrating 
assessment and learning and, secondly, the need to develop technological tools that facilitate this relationship and 
integration. In this paper, firstly we describe the EvalCOMIX® web service and then we present the opinions of 
university tutors and students that have used this web service in their courses. We conclude that EvalCOMIX® is actually 
more than a just a web-based programme for assessment. Through its use, on the one hand, it can encourage student 
participation in the assessment process, by selecting or defining criteria, building tools and processes used in 
self-assessment and peer assessment. In addition, students receive valuable and relevant information about their 
performance and progress, so that improvements can be incorporated both in their learning process and the results they 
achieve. 
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1. INTRODUCTION 

For many years assessment strategies and practices have emphasized on the one hand the importance of 
integrating assessment and learning and, secondly, the need to develop technological tools that facilitate this 
relationship and integration. 

To support the integration of learning and assessment the development of the EvalCOMIX® tool began in 
2010. It has subsequently transformed into a web-based programme that both enables and facilitates the 
implementation of the concept of assessment as learning and empowerment. 

In this chapter, firstly we describe this web programme and then we present the opinions of university 
tutors and students that have used the EvalCOMIX® programme in their courses. 


2. FROM ASSESSMENT OF LEARNING TO ASSESSMENT AS 
LEARNING AND EMPOWERMENT 

Traditionally the assessment of learning has been characterized by maintaining a separation between teaching 
and learning (Falchikov, 2005, p. 82) and employing a narrow range of systems which do not always reflect 
the objectives of the curriculum and often fail to specify the marking criteria, therefore giving little or no 
power to students, missing the opportunity to make them responsible for their own learning process and, 
among other things, causing adverse feelings and lasting negative consequences. 

Faced with this situation, common in most educational contexts, many authors with innovative 
approaches to their practices and contributions are now proposing alternative scenarios that increasingly 
place the focus of assessment on students’ progress and learning rather than on just providing a final grade. 

In relation to this we would highlight the contributions of; Carless (2007) on learning-oriented 
assessment; Rodriguez-Gomez and Ibarra-Saiz (2011) who expand this concept whilst emphasising the 
possibilities and importance of electronic forms of assessment; Taras (2010), who stresses the role of 
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self-assessment; Nicol, Thomson & Breslin (2014) who focus on the value of peer assessment; the concept of 
sustainable assessment as expressed by Boud (2000); Boud & Molloy (2013) who concentrate on feedback to 
students or Price, Rust, O'Donovan, Handley & Bryant (2012) with their contributions regarding the need for 
both tutors and students to receive training in assessment processes. 

The authors of this article are currently focused on the development and application of the concept of 
assessment as learning and empowerment (Rodriguez-Gomez and Ibarra-Saiz, 2015; Ibarra-Saiz and 
Rodriguez-Gomez, 2016) characterized by three main challenges: achieving the participation of students in 
the assessment process, incorporating self-assessment, peer assessment and shared assessment; feedforward, 
understood as strategies that provide proactive information on students’ progress and results so that they can 
participate in their improvement; and high quality tasks, ie, challenging tasks that are motivational and 
related to daily life. The implementation of these three challenges allows university students to self-regulate 
their learning process and provides empowerment within their personal, professional and working 
environments. 


3. THE NEED FOR AUTOMATION IN ASSESSMENT PROCESSES 

Currently, within the contexts experienced by the authors, the process of teaching and learning takes place in 
classrooms with large numbers of students, despite what recent legislative changes intended to achieve. 
Furthermore, it covers both knowledge and skills and does not take place only within the classroom but in 
some cases is developed entirely outside them. These are factors that also imply a change in the conditions 
and characteristics of the relevant assessment. Within universities, the non-contact nature of some or all 
subject areas is a fact that requires the use of campus-based or virtual platforms and therefore appropriate and 
valuable tools are needed to undertake assessment through these platforms and which incorporate the latest 
thinking on assessment practice. 

Assessing skills and knowledge involves using different strategies and undertaking assessment at 
different times with different objectives and criteria. This calls for an effort that has produced and is 
producing some unease and insecurity among university tutors, who face a demanding environment and who 
often do not have the most convincing answers. In this situation technology should facilitate the assessment 
process. It should support assessment, encouraging the staging of the progress that is taking place through the 
assessment process. 

According to the concept of assessment as learning and empowerment, these technological tools must be 
integrated within high quality tasks, encourage the participation of students in their own assessment process 
and provide useful and relevant information on their progress so they can take appropriate decisions in order 
to improve their work and performance. It is with the express intention of connecting and consolidation these 
various propositions that the EVALfor Research Group has been developing the by EvalCOMIX® 
web-based programme which we refer to in the following sections. 


4. HOW DOES EvalCOMIX® WORK? 

EvalCOMIX® is a web-based programme (http://evalcomix.uca.es) that supports the creation and 
implementation of assessment tools (rubrics, grading scales, mixed instruments etc.) and their use in 
assessment process both by tutors and students. Consecuently, it demands their active participation in the 
assessment process. 

Below we present how this programme works once installed on a virtual campus platform and we focus 
on answering the following questions: how can effective assessment instruments be designed?, how should 
the roles of assessors be assigned to tutors and students?, how can the weighted scores of the assessments 
provided by EvalCOMIX® be interpreted?, how can the results of the assessments be analysed? 
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4.1 Designing Assessment Instruments 


Decisions on issues such as the design of assessment instruments, specifying the criteria or the selection of 
participatory assessment methods are all included within the programme (Rodriguez-Gomez and Ibarra-Saiz, 
2016). In this section we discuss how to design assessment tools using the EvalCOMIX® programme. 

To design appropriate assessment tools it is first necessary to access the Tool Management section. From 
that screen New Instrument is selected. A list of the possible tools that can be built with EvalCOMIX® 
(Figure 1) is then displayed. 



Figure 1. Options and assessment tools provided by EvalCOMIX® 

EvalCOMIX® guides the process of construction of each instrument by offering options about which 
decisions must be taken (Figure 2). Before building the tools it is preferable to plan and determine the 
weighting that each aspect will have within the overall assessment. By default, EvalCOMIX® assigns a 
pro-rata weighting, but the percentages of all of the elements assessed (dimensions and attributes) can be 
changed as required. Once a new instrument has been created, it is automatically added to the full list of 
instruments available for tutors. 



Figure 2. Screen for the construction of a rubric 

4.2 Assigning the Roles of Assessor 

One advantage of EvalCOMIX® is that the role of assessor can be undertaken by both the tutors and the 
students, through self-assessment, peer review, using all or some of the three modes of assessment. To do this 
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each assessor is assigned the assessment tool they will use (Figure 3). The instrument can be the same or 
different for each of the assessors. Peer review can be anonymous or public. Furthermore, this type of 
assessment can be carried out by groups or by individual students. 


Assessment Planning (?) 
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Figure 3. Assigning roles in assessment and weighting 


4.3 Considerations on the Issue of Weighting 

The weighting of the grades using EvalCOMIX® is achieved by two means. Firstly, for the elements of 
assessment in which the weighting of the instrument used in each type of assessment is specified (Figure 4) 
and, secondly, for the assessment tools where the weighting is provided for each component and, within 
them, to each of the separate elements, scored between a minimum score of 0 and maximum of 100. The 
following figure shows an excerpt from a rubric in which the weighting of the two elements that are assessed 
(format and content) are specified as well as the specific elements that are being assessed. 
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Figure 4. Example of weighting for each component and element in a rubric created using EvalCOMIX® 
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4.4 Analysis of Results 

Once the assessment process is complete (construction of instruments and assessment by the tutors and 
students) EvalCOMIX® (Figure 5) enables both tutors and students to see all the assessments that have been 
completed and who has completed them following the appropriate assessment procedures. Tutors can also get 
an overview of the gradings in simple diagrams, where the maximum, minimum and median scores are 
shown. Assessment results can be analysed by student group, class and assessment methods used. 


Modality 

Teacher Assessment - TA 
Self Assessment - SA 
Peer Assessment - PA 


Grade 

80/100 K 

93/100 c \X 

75/100 c X99/IOO X 


Weighing on final grade 

60% 

10% 

30% 


©EvalCOMIX grade: 83.40 / 100 



Figure 5. Grades achieved by assessment method and weighting and representation using box and whisker plots of scores 

on self-assessment and peer assessment of a class 


5. WHAT DO USERS SAY ABOUT EVALCOMIX®? 

Two of the main features of the EvalCOMIX® programme are that both tutors and students can use it (in 
groups and individually) and that it provides information on the results achieved for all of the elements 
assessed. This aligns with the concept of assessment as learning and empowerment as it facilitates the 
participation of students in their own assessment process and provides useful and timely information to 
students on their performance. 

Specifically designed questionnaires were employed to obtain the opinion of tutors and students about a 
variety of issues after they had used EvalCOMIX® in their academic programmes, and the results are 
presented in the following sections. 

5.1 The tutors’ Views 

During the academic year 2014/2015 a total of 65 tutors from Latin American universities completed a 
questionnaire regarding their opinions after employing the EvalCOMIX® programme. Table 1 shows the 
distribution of these tutors by gender, their university and subject area. 

Table 1. Distribution of the sample of tutors by Gender, University and Subject area 


Gender 


Male 

18 

33.85 

Female 

43 

66.15 

y 

PUCESI (Ecuador) 

31 

47.69 

UCR (Costa Rica) 

27 

41.54 

UNA (Nicaragua) 

7 

10.77 

rea 



Arts and Humanities 

10 

15.38 

Science 

12 

18.46 

Health Sciences 

10 

15.38 

Social Sciences 

21 

32.31 

Engineering and Architecture 

12 

18.46 
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In Figure 6 we see how, for most tutors, the EvalCOMIX® programme provides a simple process for the 
construction of assessment instruments which can be easily modified and shared with others in a way that 
encourages a collaborative approach to the design and creation of assessment instruments. 



Create instruments Modify instruments Share instruments Collaborate in the 
easily design and creation of 

instruments 

■ 1 (None) ■ 2 *3 "4 u5 "6 (A lot) 


Figure 6. Tutors’ opinions on the constmction of instruments using the EvalCOMIX® programme 


When tutors were asked about the usefulness and benefits of EvalCOMIX® (Figure 7), their opinion was 
in the main positive. They considered it to be a programme that simplifies the assessment process for tutors, 
is applicable to any subject area, improves the competence level of tutors in dealing with assessment, 
encourages innovation within the university and they believe it should be more widely used in all 
universities. Overall, 87.69% of tutors expressed a high degree of satisfaction with EvalCOMIX®. 
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Figure 7. Tutors’ opinions on the usefulness and benefits of the EvalCOMIX® programme 

5.2 The students’ Views 

The students surveyed were undergraduates taking degrees in Business Administration and Management 
(BAM) and Finance and Accounting (F&A) who during the academic years 2012-13 (73 students) and 
2013-14 (92 students) used EvalCOMIX® within their courses. Their opinions produce interesting results 
(Rodriguez-Gomez and Ibarra-Saiz, 2016). The results obtained by asking students of BAM and F&A on 
using the EvalCOMIX® programme, on self-assessment and peer assessment are presented in Figure 8. 

It shows that students show a higher degree of agreement to the issue that EvalCOMIX® was "useful to 
provide prior knowledge of the criteria and assessment tools, as well as the elements being assessed" (M = 
4.65, SD = 1.09). They also consider using EvalCOMIX® as "simple and easy" (M = 4.61; SD = 1.10); 
"useful because the information that was provided enables subsequent tasks or activities to be improved" 
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(M = 4.50, SD = 1.14); "motivational because the feedback from tutor assessment, self-assessment, and peer 
assessment was available simultaneously and rapidly" (M = 4.49, SD = 1.17) and, finally, it proved to be a 
"friendly and intuitive environment" (M = 4.24; SD = 1.19). 


70 


60 

50 

40 

30 

20 

10 

0 



miniiwiimimi 


Simple and easy User friendly and Useful to know Motivating Useful for 

intuitive about improvement 


■ Disagree ■ Slightly agree ■ Generally agree "Totally agree 


Figure 8. The opinions of students on using the EvalCOMIX® programme for self-assessment and peer assessment 


6. CONCLUSION 

In the previous section we have presented the very positive results on the opinions of tutors and students 
regarding the use of EvalCOMIX® in the assessment process. 

These positive opinions focus on the one hand on the programme’s characteristics, such as its simplicity 
and user friendly environment. The tutors further indicate that is applicable to all subject areas and it 
encourages collaboration in creating instruments and modifying them to adapt to different situations. The 
opinions of users also indicated that EvalCOMIX® promotes transparency in assessment, specifying the 
assessment criteria and making the instruments public prior to their implementation. Furthermore the 
feedback from students was particularly positive about how motivating it was and how they felt involved as a 
result of the speed with which they received feedback on their performance and progress. In terms of the 
concept of assessment as learning and empowerment it is important to emphasize the value of the information 
provided by the self-assessment, peer assessment and tutor evaluation via EvalCOMIX® which enables them 
to improve their subsequent work. 

We conclude that EvalCOMIX® is actually more than a just a web-based programme for assessment. 
Through its use, on the one hand, it can encourage student participation in the assessment process, by 
selecting or defining criteria, building tools and processes used in self-assessment and peer assessment. In 
addition, students receive valuable and relevant information about their performance and progress, so that 
improvements can be incorporated both in their learning process and the results they achieve. 

EvalCOMIX® is a web programme that promotes learning and assessment in a fully integrated way but, 
to achieve this it is vital we fully understand the concept of assessment. We will only be able to fully exploit 
the potential of EvalCOMIX® if we are able to; introduce innovations in assessment; ensure all assessment is 
fully transparent, proposing and agreeing in advance the activities and elements that will be assessed and the 
criteria that will be applied and agreeing in advance the products and actions to be evaluated; enable the full 
participation of students in the evaluation process; provide both tutors and students with feedforward on 
student performances; and fully integrate the use of of high quality tasks within assessment. 
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